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5 STATEMENT OF GOVERNMENT INTEREST 

6 The invention described herein may be manufactured by or for 

7 the Government of the United States of America for Governmental 

8 purposes without the payment of any royalties thereon or 

9 therefore . 

10 

11 CROSS-REFERENCE TO RELATED APPLICATIONS 

12 This patent application is co-pending with related patent 

13 applications entitled NEURAL DIRECTORS (U.S. Patent Application 

14 Ser. No. 09/436,957), NEURAL SENSORS (U.S. Patent Application 

15 Ser. No. 09/436,956), STATIC MEMORY PROCESSOR (U.S. Patent 

16 Application Ser. No. 09/477,638), DYNAMIC MEMORY PROCESSOR (U.S. 

17 Patent Application Ser. No. 09/477,653), MULT I MODE INVARIANT 

18 PROCESSOR (U.S. Patent Application Ser. No. 09/641,395) and A 

19 SPATIAL IMAGE PROCESSOR (Attorney Docket No. 77346) , by the same 
2 0 inventor as this patent application. 

21 

22 BACKGROUND OF THE INVENTION 

23 (1) Field of the Invention 

24 The invention relates generally to the field of color 

25 sensors and more particularly to color sensors having neural 

26 networks with a plurality of hidden layers, or multi-layer neural 



1 networks, and further to a new neural network processor for 

2 sensing color in optical image data. 

3 (2) Description of the Prior Art 

4 Electronic neural networks have been developed to rapidly 

5 identify patterns in certain types of input data, or accurately 

6 to classify the input patterns into one of a plurality of 

7 predetermined classifications. For example, neural networks have 

8 been developed which can recognize and identify patterns, such as 

9 the identification of hand-written alphanumeric characters, in 

10 response to input data constituting the pattern of on and off 

11 picture elements, or "pixels", representing the images of the 

12 characters to be identified. In such a neural network, the pixel 

13 pattern is represented by, for example, electrical signals 

14 coupled to a plurality of input terminals, which, in turn, are 

15 connected to a number of processing nodes, each of which is 

16 associated with one of the alphanumeric characters which the 

17 neural network can identify. The input signals from the input 

18 terminals are coupled to the processing nodes through certain 

19 weighting functions, and each processing node generates an output 

20 signal which represents a value that is a non-linear function of 

21 the pattern of weighted input signals applied thereto. Based on 

22 the values of the weighted pattern of input signals from the 

23 input terminals, if the input signals represent a character that 

24 can be identified by the neural network, the one of the 

25 processing nodes associated with that character will generate a 
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1 positive output signal, and the others will not. On the other 

2 hand, if the input signals do not represent a character that can 

3 be identified by the neural network, none of the processing nodes 

4 will generate a positive output signal. Neural networks have 



5 been developed which can perform similar pattern recognition in a 

6 number of diverse areas . 

7 The particular patterns that the neural network can identify 

8 depend on the weighting functions and the particular connections 

9 of the input terminals to the processing nodes. The weighting 

10 functions in, for example, the above-described character 

11 recognition neural network, essentially will represent the pixel 

12 patterns that define each particular character. Typically, each 

13 processing node will perform a summation operation in connection 

14 with values representing the weighted input signals provided 

15 thereto, to generate a sum that represents the likelihood that 

16 the character to be identified is the character associated with 

17 that processing node. The processing node then applies the non- 
18 linear function to that sum to generate a positive output signal 

19 if the sum is, for example, above a predetermined threshold 

20 value. Conventional non-linear functions which processing nodes 

21 may use in connection with the sum of weighted input signals is 

22 generally a step function, a threshold function, or a sigmoid, in 

23 all cases the output signal from the processing node will 

24 approach the same positive output signal asymptotically. 

25 
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1 Before a neural network can be useful, the weighting 

2 functions for each of the respective input signals must be 

3 established. In some cases, the weighting functions can be 

4 established a priori. Normally, however, a neural network goes 

5 through a training phase, in which input signals representing a 

6 number of training patterns for the types of items to be 
classified, for example, the pixel patterns of the various hand 
written characters in the character- recognition example, are 



7 



8 



9 applied to the input terminals, and the output signals from the 

10 processing nodes are tested. Based on the pattern of output 

11 signals from the processing nodes for each training example, the 

12 weighting functions are adjusted over a number of trials. After 

13 the neural network has been trained, during an operational phase 

14 it can generally accurately recognize patterns, with the degree 

15 of success based in part on the number of training patterns 

16 applied to the neural network during the training stage, and the 

17 degree of dissimilarity between patterns to be identified. Such 

18 a neural network can also typically identify patterns that are 

19 similar, but not necessarily identical, to the training patterns. 

20 One of the problems with conventional neural network 

21 architectures as described above is that the training 

22 methodology, generally known as the "back -propagation" method, is 

23 often extremely slow in a number of important applications. In 

24 addition, under the back-propagation method, the neural network 

25 may result in erroneous results that may require restarting of 
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1 training. Even after a neural network has been through a 

2 training phase, confidence that the best training has been 

3 accomplished may sometimes be poor. If a new classification is 

4 to be added to a trained neural network, the complete neural 

5 network must be retrained. In addition, the weighting functions 

6 generated during the training phase often cannot be interpreted 

7 in ways that readily provide understanding of what they 

8 particularly represent:. 

9 Edwin H. Land's Retinex theory of color vision is based upon 

10 "three color" experiments performed before 1959. A simple 

11 "mishap" showed that three colors were not always required to see 

12 accurate color. Land used a short and long record of brightness 

13 data (black and white transparencies) to produce color perceived 

14 by human eyes and not by photographic means. He demonstrated a 

15 perception of a full range of pastel colors using two very 

16 similar in color light sources such as yellow, at 579 nm and 

17 yellow orange, at 599 nm ("Experiments in Color Vision", Edwin H. 

18 Land, Scientific American, Vol. 200 No. 5, May 1959). Land found 

19 that in some two record experiments all colors present were not 

20 perceived. Although Land demonstrated that two records provided 

21 color perceptions, he constructed his Retinex theory upon three 

22 records such as his long, medium and short records (An 

23 Alternative Technique for the Computation of the Designator in 

24 the Retinex Theory of Color Vision", Edwin H. Land, Proceedings 

25 of the National Academy of Sciences, Vol. 83, 1986). The 
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1 invention herein is related to human color perception discovered 

2 during Land's color vision experiments as reported in 1959, 

3 The "Trichromatic" theory in human color vision has been 

4 accepted on and off since the time of Thomas Young in 1802 (A 

5 Vision in the Brain", S. Zeki, Blackwell Scientific Publishing, 

6 1993) . Still and video electronic camera designs are correctly 

7 based upon the trichromatic theory but the current designs are 

8 highly subjective to color error reproduction due to changes in 

9 the ambient light color temperatures and color filtrations. The 

10 device in this invention senses color using a new "bichromatic" 

11 theory, which includes a mechanism that insures color constancy 

12 over a large range of ambient color temperatures. The use of two 

13 lightness records as used by Land in 1959 is one key to this 

14 invention. 

15 The bichromatic theory is based upon an interpretation of a 

16 biological color process that occurs in the eyes and brain of 

17 humans and in some animals. The bichromatic theory is defined as 

18 a system that functions together under the following assumptions, 

19 accepted principles and rules of procedure, for which FIGS. 4A 

20 and 4B are provided for support: 

21 (l) The system is a color sensing retina. There are at 

22 least two photo transducers in each pixel space in the retina, 

23 shown in FIG. 4B as TR (HI) and TR(LO) . 

24 (2) The two photo transducers sense the color of the light 

25 at each pixel's position in a scene of color focused on the 
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1 retina. Each of the at least photo transducers contains a 

2 different spectral response and the wavelength difference between 

3 the peaks of a pair of these responses is called the waveband or 

4 the spectral bandwidth of the two photo transducers. 

5 (3) The two photo transducers have overlapping spectral 

6 logarithmic responses where their slopes are opposing each other 

7 as indicated in FIG. 4A. 

8 (4) The photo transducers have at least two controlled gain 

9 amplifiers (CGA) and at least two common controlling circuits. 

10 There is one controlled gain amplifier for each photo transducer 

11 where each of the at least two common controlling circuits 

12 controls the controlled gain amplifiers for all the photo 

13 transducers of the same spectral response. 

14 ( 5 ) T he highest energy value in the retina, or the peak 

15 energy from a photo transducer of a specific spectral response, 

16 controls the output of the common controlling circuits that 
normalize the logarithmic response of all photo transducers with 



17 
18 
19 
20 



the same spectral response. Thus, it is always the peak energy 
photo transducer no matter its position in the retina that 
controls the common mode gain. The peak response of a photo 

21 transducer is relative to the best matched wavelength of energy 

22 for all wavelengths of light impinging on the color retina. 

23 Therefore, each photo transducer will be continuously normalized 

24 to the peak photo transducer signal in response to changes in 

25 ambient lighting. 
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1 (6) In a general discussion herein a normalized photo 

2 transducer or a normalized pixel includes the controlled gain 

3 amplifier as part of its response. A photo transducer sensing 

4 the peak energy or a peak energy sensing photo transducer will 

5 only be called as such thus a normalized photo transducer will 

6 not specifically include a peak energy sensing photo transducer. 

7 (7) There are three color coordinates called hue, lightness 

8 and saturation. Three degrees of freedom are required to 

9 categorize all combinations of color attributes. Two points in a 

10 two dimensional space can be connected by a line. Combinations 

11 of positions of these two points in space can produce at least 

12 three families of lines in the two dimensional space. The line 

13 families are horizontal, vertical and sloped. FIG. 4A shows a 

14 two dimensional graph of the responses of two normalized photo 

15 transducers. A straight line on the graph may represent the two 

16 output values of the normalized photo transducers for a specific 

17 input light condition. The graph coordinates are light 

18 wavelength for the horizontal axis and signal in a natural log 

19 scale for the vertical axis. Output values of the two normalized 

20 photo transducers can be represented by three families of lines. 

21 (8) The response "curve" of a normalized photo transducer 

22 output signal for a normalized light energy input is shown as a 

23 straight line, from the maximum response at its wavelength, down 

24 to the bottom at the opposite side of the graph. Each response 

25 curve of the normalized photo transducers has opposing slopes 
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1 that cross each other. A normalized photo transducer response 

2 over the waveband is given as TR(b)=ce" kx , where: x equals the 

3 wavelength position in the normalized waveband relative to the 

4 maximum response of the photo transducer, i.e., (0 to 1); c, the 

5 conversion constant, equals one for a normalized light energy, 

6 or, alternately, an integrated CGA value; k equals approximately 

7 10; and b is the high or low transducer. The output signal level 

8 is symbolized by El for the low wavelength normalized photo 

9 transducer and E2 for the other. 

10 (9) A broad constant energy spectrum of visible light 

11 relative to its color temperature "flattens" its spectral energy 

12 curve as the color temperature increases from a deep red at 1000 

13 °K to a "slightly bluish" white at 10,000 °K. Thus, when the 

14 peak energy photo transducers normalize the retina's response, 

15 the results are equivalent to "whitening" the pixel's responses 
in the waveband of sensible colors. In other words, possibly 



16 

17 



20 



different energies near che wavelength of the maximum 
sensitivities of the peak energy transducers contain 



18 

19 approximately equal spectral energies at the output of the 

respective controlled gain amplifiers. This process develops a 



21 color constancy in ambient lights of different color 

22 temperatures . 

23 (10) A family of horizontal lines can represent the 

24 normalized photo transducer responses to a broadband family of 

25 white light from bright through gray to dark. Example 1 on the 
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graph is a representation of this family. A family of vertical 
lines can represent a family of wavelengths in the waveband. 
Example 2 on the graph is a representation of the wavelength of a 
monochromatic light source. Families of sloped lines, from a 
horizontal position to a vertical position, closely represent a 
morphing from "white" to a monochromatic light. A change from 
white light to a light of a pure color is along the axis for the 
color attribute of saturation. Example 3 on the graph is a 
representation of a pastel color. The three families of lines 
are closely mapped to the three color coordinates of hue, 
lightness and saturation, but not with an exact one to one 
correlation. A combination of either set of three dimensions of 
color attributes can be mapped into the other. The two response 
values of a normalized pixel can represent a line that can move 
in combinations of the three coordinate ways to represent exact 
changes in lightness, hue and saturation of colors. 

(11) The output values of a normalized pixel, in response 
to a monochromatic light, shall exhibit proportional photo 
transducer output values of El and E2 that are relative to the 
wave length of the light in the waveband between the two photo 
transducers. In the case where there is a broad spectrum of 
light illuminating an object, the different reflective bands of 
light relative to the wavelength responses of the normalized 
pixel will produce photo transducer output values in proportional 



1 to values that would be generated by a colored light of the 

2 perceived color. 

3 (12) Changing the pixel ! s response from straight lines to 

4 curved lines on the logarithmic scale does not change the two 

5 point families of lines but it will change the form of the 

6 mapping between the two different color attributes. 

7 (13) There is another control mode that increases the 

8 dynamic range of the sensibility to light of all photo 

9 transducers in the retina. This control sums the energy of all 

10 spectral responses to adjust an iris to maintain a constant 

11 energy to the retina under varying environmental lighting 

12 intensities. 

13 (14) This Dichromatic theory projects that human color 

14 vision may not be as commonly believed. The human retina 

15 contains three color cones to sense three different wavelengths 

16 of light, which may be used as two color pairs such as a 

17 blue-green pair and a red-green pair. Each color pair is 

18 processed in the visual cortex to map colors that can be 

19 associated to the visual space of an object in a scene. The two 

20 color pairs and processing will produce a wide range of colors 

21 sensed and a wide range of color constancy. Edwin H. Land's 

22 pre- 1959 experiments using two black and white transparencies and 

23 two color filters produced a perception of color. The color 

24 perception and constancy occur because the brightest area of one 

25 of the projected transparencies normalizes the response of the 
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appropriate set of human color cones to the specific color 
projected and the same occurs for the other transparency. The 
normalized human retina now sees varying ratios of brightness 
(energy) over the visual scene, which produces the perception of 
colors of light for the specific color temperatures of natural or 
artificial light. The bichromatic theory of color is an 
integration of the above fourteen theorems that together define 
the workings of color perception and color constancy. 

SUMMARY OF THE INVENTION 

It is therefore an object of the invention to provide a new 
and improved neural network color sensor. 

It is a further object to provide a neural network color 
sensor in which the weighting functions may be determined a 
priori . 

Another object of the present invention is to provide a 
neural network color sensor, which can be trained with a single 
application of an input data set. 

In brief summary, the color sensor generates color 
information defining colors of an image, comparison of colors 
illuminated under two or more light sources and boundaries 
between different colors. The color sensor includes an input 
section, a color processing section, a color comparison section, 
a color boundary processing section and a memory processing 
section. The input section includes an array of transducer 
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1 pairs, each transducer pair defining one of a plurality of pixels 

2 of the input section. Each transducer pair comprises at least 

3 two transducers, each generating an output having a peak at a 

4 selected color, the selected color differing as between the two 

5 transducers, and each transducer having an output profile 

6 comprising a selected function of color. The color processing 
section includes a plurality of color pixel processors, each 
receiving the outputs from the two transducers comprising the 
transducer pair associated with a pixel. In response, the color 



7 
8 



10 



processing section generates a color feature vector 

11 representative of the brightness of the light incident on the 

12 pixel and a color value corresponding to the ratio of outputs 

13 from the transducers comprising the transducer pair associated 

14 with the pixel. The color boundary processing section generates 

15 a plurality of color boundary feature vectors, each associated 

16 with a pixel, each representing the difference between the color 

17 value generated by the pixel color processor for the respective 
pixel and color values generated by the pixel color processor for 



18 

19 pixels neighboring the respective pixel. 

20 The color boundary sensor produces object shape feature 

21 vectors from a function of the differences in color. This color 

22 boundary sensor can sense a colored object shape in a color 

23 background where a black and white sensing retina could not 

24 detect differences in lightness between the background and the 

25 object. The color comparator processor can measure and compare 
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the reflective color of two objects, even when each object is 
illuminated by two lights of different color temperatures. 
Thememory processor section provides a process to recognize a 
color, a boundary of color and a comparison of colors. 

BRIEF DESCRIPTION OF THE DRAWINGS 
A more complete understanding of the invention and many of 
the attendant advantages thereto will be readily appreciated as 
the same becomes better understood by reference to the following 
detailed description when considered in conjunction with the 
accompanying drawings wherein corresponding reference characters 
indicate corresponding parts throughout the several views of the 
drawings and wherein: 

FIG. 1 is a functional block diagram of a color sensor 
constructed in accordance with the invention; 

FIG. 1A is an expanded view of a transducer pair; 

FIG. 2 is a functional block diagram of a color processor, 
which is useful in the color sensor depicted in FIG. 1; 

FIG. 3 is a functional block diagram of a color boundary 
processor, which is useful in the color sensor of in FIG. 1; 

FIG. 4A is an example of the responses of two normalized 
photo transducers used in the color sensor; and 

FIG. 4B is a schematic illustration of the theorems defining 
the workings of the color sensor. 



1 DESCRIPTION OF THE PREFERRED EMBODIMENT 

2 FIG. 1 is a functional block diagram of a color sensor 10 

3 constructed in accordance with the invention. By way of 

4 background, the color sensor 10 operates in accordance with a 

5 "bi- chromatic" mechanism of color recognition, which is theorized 

6 as being similar to the way in which human beings see and 

7 recognize color. In the conventional "tri-chromatic" color 

8 recognition mechanism, any color light, either reflected or 

9 incidental, can be generated combining three different color 
10 illuminations. In the reverse, i.e., color recognition, any 
n input color can be represented or analyzed as a combination of 

12 three colors, i.e., base colors. Accordingly three transducers, 

13 each sensing one of the base three colors, can be used to 

14 determine the contribution of each of the base colors in the 

15 input color. In the bi-chromatic mechanism, colors can be 

16 distinguished using two color transducers, which have peak 

17 sensitivity at different colors, and provide a known output 

18 signal response as a function of the input color. The color 

19 sensor 10 determines, for an input image, the distribution of 

20 colors over the image, using two color transducers to identify 

21 the color at each point (that is, for each pixel or picture 

22 element) in the image. The color boundary process produces 

23 object shape features relative to the boundaries between 

24 different colors. The color comparator process produces 

25 comparative features relative to a "true reflective color" in 
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ambient lights of different color temperatures. The reading of a 
"true reflective color" in an ambient light of a color 

3 temperature and the reading of the same "true reflective color" 

4 in an ambient light of a second color temperature is a process 

5 that mimics human color constancy. 

6 With reference to FIG. 1, the color sensor 10 includes an 
input section 11, a color processing section 12 and a color 
boundary processing section 13, a color comparison processor 19 

9 and a memory processor 29. The color processing section 12 and a 

10 color boundary processing section 13 both generate color and 

11 color boundary feature vectors, which may be provided to, for 

12 example, a memory processing section 14. The input section 11 
receives an image of an object and generates, for each point, or 
pixel, color information signals representative of the color at 

15 the particular point of the image. The input section 11 includes 

16 a "retina" 15, which comprises an array of transducer pairs 15(1) 

17 through 15 (M) (generally identified by reference numeral 15 (m) 
is and shown in the expanded view of FIG. 1A) , which define the 

19 pixels of the image. Each transducer pair comprises two 

20 transducers, which have output peaks at two different frequencies 

21 and which provide a predetermined output value as a function of a 
color wave band. Preferably, all of the pixels will have one 
transducer 15 (m) (1) which has a peak output at one frequency 
identified as 1 and the second transducer 15 (m) (2) having a peak 
output at a second frequency identified as 2. The input section 
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23 
24 

25 
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11 further includes a lens 26, which focuses an image of the 
object onto the retina 15, and an iris 17, which controls the 
intensity of light incident on the retina 15. 

The color processing section 12 uses the color information 
signals from the input section to generate, for each pixel, a 
local color feature vector representative of the color of the 
pixel. The color processing section 12 consists of a color 
processor array 20 and a feature fusion network array 23. The 
9 structure and operation of the color processing section 12 will 

10 be described in detail below in connection with FIG. 2. 

11 Similarly, the color boundary processing section 13 generates, 

12 for each pixel, a local color gradient feature vector that 

13 represents the gradient of the color at the pixel. The structure 

14 and operation of the color boundary processing section 13 will be 

15 described in detail below in connection with FIG. 3. The memory 

16 processor 29 is as described in STATIC MEMORY PROCESSOR, U.S. 

17 Patent Application Ser. No. 09/477,638. The parallel memory 

18 processors 16 and 18 are as described for the memory processor of 

19 the MULTIMODE INVARIANT PROCESSOR (U.S. Patent Application Ser. 



20 



No. 09/641,395). The multi-mode invariant image processor. 



21 without its input sensor, is used for both parallel memory 

22 processors 16 and 18. The possible multiple outputs of the 

23 parallel memory processor 18 are the colored input object (s) 

24 classifications. The output vector array of the parallel memory 

25 processor 16 is a Positional King Of the Mountain (PKOM) array 
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1 mapped to the pixels 15 (m) in the retina, which becomes a map of 

2 color classifications of each pixel. It is noted that the PKOM 

3 array is a neural network array internal to the parallel memory 

4 processor 16 and the remaining neural circuits to the normal 

5 output of the MULTIMODE INVARIANT PROCESSOR are not used. The 

6 memory processor 29 is a static memory processor and provides an 

7 output classification as a degree of color comparison. 

8 The local color feature vectors and the local color gradient 

9 feature vectors generated for all of the pixels are processed by 

10 the processing section 14 to, for example, classify the image 

11 into one of a plurality of image classes. The processing section 

12 14 may comprise any of a plurality of processing elements for 

13 processing the vectors generated by the color processors 12, 13 

14 and/or 19. 

15 FIG . 2 is a functional block diagram of color processing 

16 section 12 and 19 as used in the color sensor of FIG. 1. With 

17 reference to FIG. 2, the color processing section 12 includes a 

18 plurality of pixel color processors 20(1) through 20 (M) , 

19 generally identified by reference numeral 20 (m) . For each color 

20 processor 20 (m) , a corresponding feature fusion network 23 (m) of 

21 color processing section 12 includes corresponding feature fusion 

22 neural directors 35(1) through 35 (M) and Multi King Of the 

23 Mountain (MKOM) 36(1) through 36 (M), generally identified by 

24 reference numerals 35 (m) and 36 (m) , respectively. The structures 

25 of all of the pixel color processors 20 (m) are similar, and so 
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1 FIG. 2 depicts the structure of only one pixel color processor 

2 and the corresponding feature fusion neural director 35 (m) and 

3 MKOM 36 (m). Each pixel color processor 20 (m) processes the 

4 outputs generated by one of the transducer pairs in the retina 

5 11. The color processing section 12 also includes a common 

6 control 21 7 which controls all of the pixel color processors 

7 2 0 (m) in parallel/ controls the iris 17 and receives pixel data 

8 from each color processor 20 (m) . 

9 Each pixel color processor 20 (m) includes controlled gain 

10 amplifier (CGA) circuits 30(m)(l), 30 (m) (2), which receive the 

11 color amplitude signals generated by the respective transducers 

12 15 (m) (1) , 15 (m) (2) . Each CGA circuit 30 (m) (1) , 30 (m) (2) 

13 generates an output adjusted by a gain control factor generated 

14 by the common control 21. The gain control factor is a function 

15 of the output of the transducer for each frequency having the 

16 highest amplitude, referred to as 15(H) (1) and 15(H) (2) . The CGA 

17 circuits 30(m)(l), 30 (m) (2) will normalize the respective outputs 

18 in relation to the highest amplitude output for their respective 

19 frequency. This allows each transducer pair 15 (m) and their 

20 respective CGA circuit 30 (m) to output differing values, which 

21 represent the color at each transducer pair 15 (m) as well as the 

22 "color temperature" of the light incident on the object or retina 

23 15. The common control 21 senses all transducer outputs for each 

24 frequency and uses the highest outputs 15(H) (1), 15(H) (2) to set 

25 each CGA circuit 3 0 (m) in the color processor 12 to the same gain 
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as the CGA circuits 30(H) (1), 30(H) (2) from the pixel (s) 15 (m) 
that sensed the highest light energy in retina 15. The 
transducers 15(H) (1), 15(H) (2), the CGA circuits 30(H) (1), 
30(H) (2) and the common control 21 operate as an automatic gain 
controlled loop normalizing the output signal at CGA circuit 
30(H) (1) . Therefore, the response of each transducer 15 (m) (1) is 
normalized at the output of each CGA circuit 30 (m) (1) relative to 
the output of CGA circuit 30(H) (1) . It is to be noted that the 
transducers 15(H) (1), 15(H) (2) need not be from the same pixel 
15 (m) , as the spectral light energy of a visual scene image at 
two separate frequencies is generally not the same everywhere on 
retina 15. 

The gain controlled output of each CGA circuit 30 (m) (1) , 
30 (m) (2) is provided to a number of elements, including a 
respective sum circuit 33 (m) , a difference circuit 32 (m) and the 
common control 21. The outputs from the CGA circuits 30 (m) (1), 
30 (m) (2) are coupled to the difference circuit, or difference 
generator 32 (m) , which generates an output vector that is 
representative of the difference between the amplitudes of the 
outputs form the CGA circuits 30 (m) (1), 30 (m) (2) . Accordingly, 
it will be appreciated that the output generated by the 
difference generator 32 (m) corresponds to the ratio of the 
amplitudes of the automatic controlled gain signals from the 
respective transducers 15(H)(1), 15(H)(2) and the respective 
pixel transducer 15 (m) outputs. 



1 As noted above, the outputs from the CGA circuits 30 (m) (1) 

2 and 3 0 (m) (2) are also coupled to a sum circuit 33 (m) . The sum 

3 circuit 33 (m) generates an output that corresponds to the sum of 

4 the amplitudes of the automatic controlled gain signal from the 

5 respective transducers 15(m)(l) and 15 (m) (2), and thus represents 

6 the brightness of the light incident on the pixel defined by the 

7 transducers . 

8 The output vector from difference circuit 32 (m) is coupled 

9 to the color boundary processor 13 (FIG. 1) . The difference 

10 vector from difference circuit 32 (m) and the brightness vector 

11 from sum circuit 33 (m) are also both coupled to a neural director 

12 35 (m) that disperses these inputs into a local color feature 

13 vector. The neural director 35 (m) is preferably similar to the 

14 neural directors as described in NEURAL DIRECTOR, U.S. Patent 

15 Application Ser. No. 09/436,957. Neural director 35 (m) is 

16 preferably established to provide an output vector with an 

17 increased dimensionality, which will aid in distinguishing 

18 between similar patterns in the input vector. 

19 The output of the neural director 35 (m) is coupled to bi- 

20 polar MKOM 36 (m), which is described in detail in STATIC MEMORY 

21 PROCESSOR, U.S. Patent Application Ser. No. 09/477,638. The bi- 

22 polar MKOM 36 (m) generates a number of positive and/or negative 

23 outputs M(l) through M(R) , generally identified by reference 

24 numeral M(r) , each of which is associated with one dimension of 

25 the feature vector input thereto. Each positive component M(r) 
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1 of the output vector can have a range of values from zero up to a 

2 maximum value, which corresponds to, or is proportional to, the 

3 maximum positive element value of the input vector. The positive 

4 outputs M(r) that are associated with an input vector component 

5 having successively lower positive values, are themselves 

6 successively lower in value, thus forming a positive ranking of 

7 the vector components. Outputs M(r) that are associated with 

8 input vector components having negative values are also ranked as 

9 negative vector components in a similar manner to the positive 

10 components. The rankings for the respective input feature 

11 vectors may be global, for all of the components of the input 

12 feature vector, or they may be localized among a selected number 

13 of preferably contiguous input feature vector components. The 

14 feature vector generated by the bi-polar MKOM 36 (m) is coupled to 

15 the memory processing section 14 . 

16 The outputs from CGA circuits 30 (m) (1) and 30 (m) (2) of all 

17 of the pixel color processors 20 (m) are also coupled to the 

18 common control 21. The common control 21 includes peak sensing 

19 circuits 40(1), 40 (2), each of which receives the output from the 

20 correspondingly- indexed CGA circuits 30 (m) (1), 30 (m) (2), and each 

21 generates an output which corresponds to the one of the outputs 

22 from the correspondingly- indexed CGA circuits 30 (m) (1), 30 (m) (2) 

23 with the largest signal value. The outputs from the peak 

24 circuits 40(1), 40(2) are also connected to control the gain of 

25 
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1 all of the correspondingly- indexed CGA circuits 30 (m) (1), 

2 30 (m) (2) . 

3 The outputs from the CGA circuits 30 (m) (1) and 30 (m) (2) of 

4 all of the color pixel processors 20 (m) are also connected to a 

5 sum circuit 41. The sum circuit 41 generates an output, which 

6 represents the sum of the outputs from all of the CGA circuits 

7 30 (m) (1), 30 (m) (2) of all of the color pixel processors 20 (m) . 

8 The output provided by the sum circuit 41 represents the total 

9 intensity or power of the light incident on the retina 15. An 

10 iris control circuit 42 uses the sum circuit 41 output to control 

11 the iris 17, which normalizes the intensity of the light on 

12 retina 15. 

13 FIG. 3 is a functional block diagram of the color boundary 

14 processor 13, which is useful in the color sensor depicted in 

15 FIG. 1. The color boundary processor 13 can sense a colored 

16 object shape in a background of a different color. A black and 

17 white sensing retina often responds to different colors as equal 

18 lightness. Therefore, it may not sense an object of one color 

19 against a different background color. As noted above, the color 

20 boundary processor 13 receives the color vector signals from the 

21 difference circuits 32 (m) of all of the pixel color processors 

22 20 (m) . Color boundary processor 13 then generates an output for 

23 each pixel 15 (m) that represents a color gradient for the pixel 

24 15 (m) . The outputs of each difference circuit 32 (m) are 

25 spatially arranged exactly in the same spatial orientation as 
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1 each associated pixel 15 (m) in retina 15. The array of 

2 difference circuit 32 (m) outputs becomes a virtual retina 55, 

3 shown in FIG. 3 to aid in the visualization of the spatial 

4 interconnections between the array of color processors 20 and 

5 color boundary processor 13 . The color boundary processor 13 

6 comprises a plurality of window difference networks 50(1) through 

7 50 (M), generally identified by reference numeral 50 (m) , each 

8 associated with one of the pixels 15 (m) and associated window 

9 57 (m) . Color boundary processor 13 further comprises a like 

10 plurality of neural directors 51 (m). 

11 Each window difference network 50 (m) receives a local window 

12 array 57 (m) of difference vectors generated by the 

13 correspondingly- indexed pixel color processor 20 (m). Each window 

14 difference network 50 (m) , in turn, generates an output vector 

15 which represents a color acceleration vector between the 

16 difference vectors provided by the correspondingly- indexed pixel 

17 color processor 2 0 (m) and color vectors for pixels within a 

18 predetermined area around the pixel 15 (m) , illustrated in FIG. 3 

19 as local window 57 (m) . Local window 57 (m) may consist of any 

20 chosen pattern of pixels surrounding pixel 15 (m) . e.g., a star 

21 pattern or a box pattern. Each neural director 51 (m) receives 

22 the color acceleration vector from the correspondingly- indexed 

23 window difference network 50 (m) . As with neural director 35 (m), 

24 each neural director 51 (m) is preferably established to provide 

25 an output local color boundary feature vector with the same or an 
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1 increased dimensionality, which will aid in distinguishing 

2 between similar patterns in the input vector. 

3 In a modification to the invention 10, each pixel can be a 

4 three transducer set 15 (m) . Each transducer of the set 15 (m) is 

5 to be matched to the response of the human retinal color cones. 

6 The three transducer set 15 (m) will produce two "transducer 

7 pairs" for each pixel 15 (m) and with two color processors 12 a 

8 color retina will be produced. The retina and two parallel memory 

9 processors 16 will sense color matched to the human color 

10 perception over a wide range of ambient lighting conditions. 

11 With reference again to FIG. 1, the local color feature 

12 vectors generated by the pixel color processing section 12, an 

13 array of color comparators 19 and the local color boundary 

14 feature vectors generated by color boundary processor 13 for all 
is of the pixels 15 (m) , are coupled to the memory processing section 

16 14. The memory processing section 14 may perform a variety of 

17 individual or combined operations in connection with the feature 
vectors input thereto, including object recognition and the like, 



based on preselected object classification patterns or the like. 



18 
19 

20 The invention provides a number of advantages. In 

21 particular, the invention provides a system for receiving an 

22 image of an object and generates, for an array of pixels of the 

23 image, color and color gradient /boundary information, in the form 
2 4 of feature vectors, which may be processed to, for example, 

25 classify the object into one of a plurality of object classes. 
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1 The system generates the color and color gradient /boundary 

2 information using only two transducers for each pixel, in 

3 accordance with a bi-chromatic color recognition scheme, with the 

4 transducers having peak responses at selected colors 1 and 2, 

5 and a known output profile as a function of color, instead of the 

6 non- color constancy process produced in accordance with the tri- 

7 chromatic color recognition scheme. 

8 It will be appreciated that numerous modifications may be 

9 made to the system 10. For example, the memory processing 

10 section 14 may perform processing in connection with comparisons 

11 generated for two images, using output color feature vectors 

12 generated either by the same color sensor 10 at two points in 

13 time, or output comparator vectors which are generated by two 

14 color sensors (the second being denoted by 11' and 12') for 

15 respective pixels 15 (m) for respective images. In that case, and 

16 with reference to FIG. 2, the color processing section 12, in 

17 particular the pixel color processors 20 (m), may provide outputs 

18 for the two images to the respective difference circuits 60 (m), 

19 61 (m) of color comparison processor 19, each of which generates a 

20 difference vector representing the difference between the 

21 difference vectors and brightness vectors generated by the color 

22 processors 12 for the respective images. The difference vectors 

23 of 60 (m) and 61 (m) are input to comparator feature fusion network 

24 array 62, which operates in a manner similar to feature fusion 

25 network array 23. Similar difference circuits (not shown) may 
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1 also be provided for the local color boundary feature vectors 

2 generated by the color difference processors 13 for the 

3 respective images. 

4 In addition, the peak detector circuits 40(1), 40(2) of the 

5 common control 21 may be replaced with summing circuits that 

6 generate a sum output for controlling the CGA circuits 30 (m) (1) , 

7 30 (m) (2) . 

8 Preferably, the iris control 42 will generally rapidly 

9 adjust the iris in response to changes in the light intensity 
levels incident on the retina 15, so as to maintain the light 



10 

11 levels incident on the transducers within a predetermined 

12 operating range. In that case, the CGA circuits 30 (m) (1), 

13 30 (m) (2) may have a relatively slower response to changes in the 

14 automatic gain control signals from the control circuit 21. 

15 These differences in response will allow the slower response of 

16 normalization via the CGA circuits to maintain a steady color 

17 constancy in a scene of rapid brightness changes. 

18 The described components of invention 10 provide the 

19 necessary components for a uniquely designed photographer's 

20 exposure and color temperature meter. A calibration of the 

21 common control network 21 provides values for exposure and color 

22 temperature data. The meter may be an independent device, i.e., 

23 a hand held meter, or it may be integrated in a camera body, 
2 4 either electronic or film, to provide automatic exposure and 

25 
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color temperature corrections. The device may also be integrated 
into color printers or printing presses as a color ink control. 

It will be apparent that variations and modifications may be 
made to the invention herein described and illustrated, by those 
skilled in the art with the attainment of some or all of the 
advantages of the invention. It is also understood that the 
color sensor described herein may be connected to the various 
devices described in the referenced patent applications, wherein 
all the devices act in concert in a manner similar to the human 
eye. Therefore, it is the object of the appended claims to cover 
all such variations and modifications as come within the true 
spirit and scope of the invention. 



